Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning

The COVID-19 pandemic has drastically changed accepted norms globally. Within the past year, masks have been used as a public health response to limit the spread of the virus. This sudden change has rendered many face recognition based access control, authentication and surveillance systems ineffective. Official documents such as passports, driving license and national identity cards are enrolled with fully uncovered face images. However, in the current global situation, face matching systems should be able to match these reference images with masked face images. As an example, in an airport or security checkpoint it is safer to match the unmasked image of the identifying document to the masked person rather than asking them to remove the mask. We find that current facial recognition techniques are not robust to this form of occlusion. To address this unique requirement presented due to the current circumstance, we propose a set of re-purposed datasets and a benchmark for researchers to use. We also propose a contrastive visual representation learning based pre-training workflow which is specialized to masked vs unmasked face matching. We ensure that our method learns robust features to differentiate people across varying data collection scenarios. We achieve this by training over many different datasets and validating our result by testing on various holdout datasets, including a real world dataset collected specifically for evaluation. The specialized weights trained by our method outperform standard face recognition features for masked to unmasked face matching. We believe the provided synthetic mask generating code, our novel training approach and the trained weights from the masked face models will help in adopting existing face recognition systems to operate in the current global environment. We open-source all contributions for broader use by the research community.

Imagery, Privacy and Ethics: An Overview of Partially Occluded Facial Biometric Analysis in the Era of Face Masks

The COVID-19 pandemic has drastically changed human lifestyles, with implications on many aspects of human life. With the proliferation of masks to combat the spread of the virus, many computer vision workflows have been inadvertently affected to varying degrees. Consequently, many research articles have been dedicated to evaluating the impact to existing facial imagery recognition problems. Several works have attempted to either extend existing facial models or develop new techniques specific to masked faces. Many new benchmark tasks have also been introduced in this subdomain. However, a detailed review of such advancements is not available for perusal in this critical area for COVID-safe protocol development. In this work, we address this issue as the first review of masked facial recognition tasks and techniques robust to masked facial images. Our motivation is to provide a central reference for automated public health and COVID-safe identification protocols while also exploring the ethical aspects of further development of such techniques.

Does a Face Mask Protect My Privacy?: Deep Learning to Predict Protected Attributes from Masked Face Images

Contactless and efficient systems are implemented rapidly to advocate preventive methods in the fight against the COVID-19 pandemic. Despite the positive benefits of such systems, there is potential for exploitation by invading user privacy. In this work, we analyse the privacy invasiveness of face biometric systems by predicting privacy-sensitive soft-biometrics using masked face images. We train and apply a CNN based on the ResNet-50 architecture with 20,003 synthetic masked images and measure the privacy invasiveness. Despite the popular belief of the privacy benefits of wearing a mask among people, we show that there is no significant difference to privacy invasiveness when a mask is worn. In our experiments we were able to accurately predict sex (94.7%), race (83.1%) and age (MAE 6.21 and RMSE 8.33) from masked face images. Our proposed approach can serve as a baseline utility to evaluate the privacy-invasiveness of artificial intelligence systems that make use of privacy-sensitive information. We open-source all contributions for reproducibility and broader use by the research community. © 2022, Springer Nature Switzerland AG.

The COVID-19 lockdowns: a window into the Earth System

The COVID-19 pandemic has caused substantial global impact. This Perspective provides insight into the environmental effects of the pandemic, documenting how it offers an opportunity to better understand the Earth System. Restrictions to reduce human interaction have helped to avoid greater suffering and death from the COVID-19 pandemic, but have also created socioeconomic hardship. This disruption is unprecedented in the modern era of global observing networks, pervasive sensing and large-scale tracking of human mobility and behaviour, creating a unique test bed for understanding the Earth System. In this Perspective, we hypothesize the immediate and long-term Earth System responses to COVID-19 along two multidisciplinary cascades: energy, emissions, climate and air quality;and poverty, globalization, food and biodiversity. While short-term impacts are dominated by direct effects arising from reduced human activity, longer-lasting impacts are likely to result from cascading effects of the economic recession on global poverty, green investment and human behaviour. These impacts offer the opportunity for novel insight, particularly with the careful deployment of targeted data collection, coordinated model experiments and solution-oriented randomized controlled trials, during and after the pandemic.